Method and apparatus for setting reference picture index of temporal merging candidate

ABSTRACT

The present invention relates to a method and apparatus for setting a reference picture index of a temporal merging candidate. An inter-picture prediction method using a temporal merging candidate can include the steps of: determining a reference picture index for a current block; and inducing a temporal merging candidate block of the current block and calculating a temporal merging candidate from the temporal merging candidate block, wherein the reference picture index of the temporal merging candidate can be calculated regardless of whether a block other than the current block is decoded. Accordingly, a video processing speed can be increased and video processing complexity can be reduced.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/408,695 filed on Jan. 18, 2017, which is a continuation of U.S.application Ser. No. 14/354,819 filed on Apr. 28, 2014, which claims thebenefit of National Stage application of International Application No.PCT/KR2012/011059, filed on Dec. 18, 2012, which claims the benefit ofpriority of Korean Patent Application No. 10-2011-0140861 filed on Dec.23, 2011, Korean Patent Application No. 10-2012-0003617 filed on Jan.11, 2012, and Korean Patent Application No. 10-2012-0147996 filed onDec. 18, 2012, the entire disclosures of which are incorporated hereinby reference for all purposes.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an image processing method andapparatus and, more particularly, to an inter-frame prediction methodand an apparatus using the method.

Related Art

A demand for images having high resolution and high quality, such as aHigh Definition (HD) image and an Ultra High Definition (UHD) image, isrecently increasing in a variety of application fields. As theresolution and quality of image data become higher, the amount of theimage data becomes relatively greater than that of the existing imagedata. For this reason, if the image data is transmitted using media,such as the existing wired/wireless broadband lines, or the image datais stored by using the existing storage medium, a transmission cost anda storage cost are increased. Image compression techniques with highefficiency can be used to solve the problems occurring as the resolutionand quality of image data becomes higher.

Image compression techniques include a variety of techniques, such as aninter-frame prediction technique for predicting a pixel value includedin a current picture from a picture anterior or posterior to the currentpicture, an intra-frame prediction technique for predicting a pixelvalue included in a current picture by using information on a pixelwithin the current picture, and an entropy coding technique forallocating a short symbol to a value having high frequency of appearanceand allocating a long symbol to a value having low frequency ofappearance. Image data can be effectively compressed, transmitted, orstored by using the image compression techniques.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a method of setting thereference picture index of a temporal merging candidate.

Another object of the present invention is to provide an apparatus forperforming a method of setting the reference picture index of a temporalmerging candidate.

In accordance with an aspect of the present invention, an inter-frameprediction method using a temporal merging candidate may include thesteps of determining the reference picture index of the temporal mergingcandidate for a current block and deriving the temporal mergingcandidate block of the current block and deriving the temporal mergingcandidate from the temporal merging candidate block, wherein thereference picture index of the temporal merging candidate can be derivedirrespective of whether other blocks except the current block have beendecoded or not. The temporal merging candidate may be derived in a unitof a coding block including the current block or in a unit of thecurrent block depending on whether the current block will use a singlemerging candidate list or not. The inter-frame prediction method mayfurther include the step of determining whether or not the current blockis a block using the single merging candidate list, wherein the singlemerging candidate list may derive and generate at least one of thespatial merging candidate and the temporal merging candidate of aprediction block based on a coding block including the prediction block.The step of determining whether or not the current block is a blockusing the single merging candidate list may include the steps ofdecoding information on the size of the current block and determiningwhether or not the information on the size of the current blocksatisfies conditions of the size of a block that the single mergingcandidate list is derived. The reference picture index of the temporalmerging candidate may be set to a fixed value. The temporal mergingcandidate may include a temporal motion vector calculated by comparing adifference between the reference picture index of a temporal mergingcandidate block (i.e., a colocated block) and the index of a picture(i.e., a colocated picture) including the colocated block with adifference between the reference picture index of the temporal mergingcandidate having the index of the fixed value and the index of thepicture including the current block. The reference picture index of thetemporal merging candidate may be set to 0.

In accordance with another aspect of the present invention, a decoderfor performing an inter-frame prediction method using a temporal mergingcandidate includes a merging candidate deriving unit configured todetermine the reference picture index of the temporal merging candidatefor a current block, derive the temporal merging candidate block of thecurrent block, and derive a temporal merging candidate from the temporalmerging candidate block, wherein the reference picture index of thetemporal merging candidate may be derived irrespective of whether otherblocks except the current block have been decoded or not. The temporalmerging candidate may be derived in a unit of a coding block includingthe current block or in a unit of the current block depending on whetherthe current block will use a single merging candidate list or not. Themerging candidate deriving unit may be configured to determine whetheror not the current block is a block using the single merging candidatelist, and the single merging candidate list may derive and generate atleast one of the spatial merging candidate and the temporal mergingcandidate of a prediction block based on a coding block including theprediction block. The merging candidate deriving unit may be configuredto decode information on the size of the current block and determinewhether or not the information on the size of the current blocksatisfies conditions of the size of a block that the single mergingcandidate list is derived, in order to determine whether or not thecurrent block is a block using the single merging candidate list. Thereference picture index of the temporal merging candidate may be set toa fixed value. The temporal merging candidate may include a temporalmotion vector calculated by comparing a difference between the referencepicture index of a temporal merging candidate block (a colocated block)and the index of a picture (a colocated picture) including the colocatedblock with a difference between the reference picture index of thetemporal merging candidate having the index of the fixed value and theindex of the picture including the current block. The reference pictureindex of the temporal merging candidate may be set to 0.

As described above, in accordance with the method and apparatus forsetting the reference picture index of a temporal merging candidateaccording to embodiments of the present invention, inter-frameprediction using a temporal merging candidate can be performed on aplurality of prediction blocks in parallel by using a temporal mergingcandidate set to a specific value or using the reference picture indexof a spatial merging candidate at a predetermined location as thereference picture index of a temporal merging candidate. Accordingly, animage processing speed can be increased, and the complexity of imageprocessing can be reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the construction of an image coder inaccordance with an embodiment of the present invention.

FIG. 2 is a block diagram showing the construction of an image decoderin accordance with another embodiment of the present invention.

FIG. 3 is a conceptual diagram illustrating an inter-frame predictionmethod using merge mode in accordance with an embodiment of the presentinvention.

FIG. 4 is a conceptual diagram illustrating inter-frame prediction usinga temporal merging candidate and the reference picture index of thetemporal merging candidate in accordance with an embodiment of thepresent invention.

FIG. 5 is a conceptual diagram illustrating a case where one codingblock is partitioned into two prediction blocks.

FIG. 6 is a conceptual diagram illustrating a method of setting thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

FIG. 7 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

FIG. 8 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

FIG. 9 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

FIG. 10 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

FIG. 11 is a conceptual diagram illustrating a method of deriving thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

FIG. 12 is a conceptual diagram illustrating a method of deriving thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

FIG. 13 is a conceptual diagram illustrating a method of deriving thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

FIG. 14 is a flowchart illustrating a method of including a temporalmerging candidate in a merging candidate list in accordance with anembodiment of the present invention.

FIG. 15 is a conceptual diagram illustrating a method of generating asingle merging candidate list by sharing all spatial merging candidatesand temporal merging candidates in a plurality of prediction blocks inaccordance with an embodiment of the present invention.

FIG. 16 is a conceptual diagram illustrating a method of generating asingle candidate list in accordance with an embodiment of the presentinvention.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, exemplary embodiments are described in detail withreference to the accompanying drawings. In describing the embodiments ofthe present invention, a detailed description of the known functions andconstructions will be omitted if it is deemed to make the gist of thepresent invention unnecessarily vague.

When it is said that one element is “connected” or “coupled” to theother element, the one element may be directly connected or coupled tothe other element, but it should be understood that a third element mayexist between the two elements. Furthermore, in the present invention,the contents describing that a specific element is “included (orcomprised)” does not mean that elements other than the specific elementare excluded, but means that additional elements may be included in theimplementation of the present invention or in the scope of technicalspirit of the present invention.

Terms, such as the first and the second, may be used to describe variouselements, but the elements should not be restricted by the terms. Theterms are used to only distinguish one element and the other elementfrom each other. For example, a first element may be named a secondelement without departing from the scope of the present invention.Likewise, a second element may also be named a first element.

Furthermore, elements described in the embodiments of the presentinvention are independently shown in order to indicate different andcharacteristic functions, and it does not mean that each of the elementsconsists of separate hardware or a piece of software unit. That is, theelements are arranged, for convenience of description, and at least twoof the elements may be combined to form one element or one element maybe divided into a plurality of elements and the plurality of elementsmay perform functions. An embodiment in which the elements are combinedor each of the elements is divided is included in the scope of thepresent invention without departing from the essence of the presentinvention.

Furthermore, in the present invention, some elements may not beessential elements for performing essential functions, but may beoptional elements for improving only performance. The present inventionmay be implemented using only the essential elements for implementingthe essence of the present invention other than the elements used toimprove only performance, and a structure including only the essentialelements other than the optional elements used to improve onlyperformance are included in the scope of the present invention.

FIG. 1 is a block diagram showing the construction of an image coder inaccordance with an embodiment of the present invention.

Referring to FIG. 1, the image coder 100 includes a motion predictionunit 111, a motion compensation unit 112, an intra-prediction unit 120,a switch 115, a subtractor 125, a transform unit 130, a quantizationunit 140, an entropy coding unit 150, an inverse quantization unit 160,an inverse transform unit 170, an adder 175, a filtering unit 180, and areference picture buffer 190.

The image coder 100 can perform coding an input picture in intra mode orinter mode and output a bit stream. The switch 115 can switch to intramode in the case of intra mode and can switch to inter mode in the caseof inter mode. The image coder 100 can derive a prediction block for theinput block of the input picture and then code the residual of the inputblock and the prediction block.

Intra mode can be defined and used as a term ‘intra-frame predictionmode’, inter mode can be defined and used as a term ‘inter-frameprediction mode’, the intra-prediction unit 120 can be defined and usedas a term ‘intra-frame prediction unit’, and the motion prediction unit111 and the motion compensation unit 112 can be defined and used as aterm ‘inter-frame prediction unit’.

An inter-frame prediction method in accordance with an embodiment of thepresent invention discloses a method of determining the referencepicture index of a temporal merging candidate. The intra-prediction unit120 can include a merging candidate deriving unit for deriving thespatial merging candidate and temporal merging candidate blocks of acurrent block and deriving a spatial merging symbol from the spatialmerging candidate block and a temporal merging candidate from thetemporal merging candidate block. A method of deriving the mergingcandidates will be described in detail later.

In the case of intra mode, the intra-prediction unit 120 can derive theprediction block by performing spatial prediction by using the pixelvalue of an already coded block near a current block.

In the case of inter mode, the motion prediction unit 111 can obtain amotion vector by searching a reference picture, stored in the referencepicture buffer 190, for a region that is most well matched with theinput block in a motion prediction process. The motion compensation unit112 can derive the prediction block by performing motion compensationusing the motion vector.

The subtractor 125 can derive a residual block by way of the residual ofthe input block and the derived prediction block. The transform unit 130can output a transform coefficient by performing transform on theresidual block. Here, the transform coefficient can mean a coefficientvalue derived by performing transform on the residual block and/or aresidual signal. In the following specification, a quantized transformcoefficient level derived by applying quantization to a transformcoefficient can also be called a transform coefficient.

The quantization unit 140 can quantize the input transform coefficientaccording to a quantization parameter and output a quantized transformcoefficient level.

The entropy coding unit 150 can perform entropy coding based on valuescalculated by the quantization unit 140 or a coding parameter valuederived in a coding process and output a bit stream based on a result ofthe entropy coding.

If entropy coding is applied, the size of a bit stream for each oftarget coding symbols can be reduced because the symbols are representedby allocating a small number of bits to a symbol having a highprobability of occurrence and a large number of bits to a symbol havinga low probability of occurrence. Accordingly, the compressionperformance of image coding can be increased by way of the entropycoding. The entropy coding unit 150 can use a coding method, such asexponential golomb, Context-Adaptive Variable Length Coding (CAVLC), orContext-Adaptive Binary Arithmetic Coding (CABAC), for the entropycoding.

In the image coder according to the embodiment of FIG. 1, a currentlycoded image needs to be decoded and stored in order to be used as areference picture because inter-prediction coding, that is, inter-frameprediction coding, is performed. Accordingly, the quantized coefficientis inversely quantized by the inverse quantization unit 160 and theninversely transformed by the inverse transform unit 170. The inverselyquantized and inversely transformed coefficient is added to theprediction block by way of the adder 175, and thus a reconstructed blockis derived.

The reconstructed block experiences the filtering unit 180. Thefiltering unit 180 can apply one or more of a deblocking filter, aSample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF) to thereconstructed block or a reconstructed picture. The reconstructed blockthat has experienced the filtering unit 180 can be stored in thereference picture buffer 190.

FIG. 2 is a block diagram showing the construction of an image decoderin accordance with another embodiment of the present invention.

Referring to FIG. 2, the image decoder 200 includes an entropy decodingunit 210, an inverse quantization unit 220, an inverse transform unit230, an intra-prediction unit 240, a motion compensation unit 250, anadder 255, a filtering unit 260, and a reference picture buffer 270.

The image decoder 200 can receive a bit stream from a coder, performdecoding on the bit stream in intra mode or inter mode, and output areconfigured image, that is, a reconstructed picture. A switch canswitch to intra mode in the case of intra mode and can switch to intermode in the case of inter mode. The image decoder 200 can obtain areconstructed residual block from the input bit stream, derive aprediction block from the reconstructed residual block, and derive ablock reconstructed by adding the reconstructed residual block and theprediction block together, that is, the reconstructed block.

The entropy decoding unit 210 can derive symbols, including a symbolhaving a quantized coefficient form, by performing entropy decoding onthe input bit stream according to a probability distribution. Theentropy decoding method is similar to the aforementioned entropy codingmethod.

If the entropy decoding method is applied, the size of a bit stream foreach of symbols can be reduced because the symbols are represented byallocating a small number of bits to a symbol having a high probabilityof occurrence and a large number of bits to a symbol having a lowprobability of occurrence. Accordingly, the compression performance ofimage decoding can be increased by the entropy decoding method.

The quantized coefficient is inversely quantized by the inversequantization unit 220 and then inversely transformed by the inversetransform unit 230. As a result of the inverse quantization and theinverse transform on the quantized coefficient, the reconstructedresidual block can be derived.

In the case of intra mode, the intra-prediction unit 240 can derive aprediction block by performing spatial prediction using the pixel valueof an already decoded block near a current block. In the case of intermode, the motion compensation unit 250 can derive the prediction blockby performing motion compensation using a motion vector and a referencepicture stored in the reference picture buffer 270.

An inter-frame prediction method in accordance with an embodiment of thepresent invention discloses a method of determining the referencepicture index of a temporal merging candidate. An intra-prediction unitcan include a merging candidate deriving unit for deriving the spatialmerging candidate and temporal merging candidate blocks of a currentblock and deriving a spatial merging symbol from the spatial mergingcandidate block and a temporal merging candidate from the temporalmerging candidate block. A method of deriving the merging candidateswill be additionally described later.

The reconstructed residual block and the prediction block are added bythe adder 255, and the added block can experience the filtering unit260. The filtering unit 260 can apply one or more of a deblockingfilter, an SAO, and an ALF to a reconstructed block or a reconstructedpicture. The filtering unit 260 can output the reconfigured picture. Thereconstructed picture can be stored in the reference picture buffer 270and used for inter-prediction.

A method of improving the prediction performance of an image coder andan image decoder includes a method of increasing the accuracy of aninterpolation image and a method of predicting a difference signal. Thedifference signal indicates a difference between the original image anda prediction image. In the present invention, a “difference signal” canbe replaced with a “residual signal”, a “residual block”, or a“difference block” depending on context. A person having ordinary skillin the art can distinguish the residual signal, the residual block, andthe difference block from each other within a range that does not affectthe spirit and essence of the invention.

In an embodiment of the present invention, a term, such as a Coding Unit(CU), a Prediction Unit (PU), or a Transform Unit (TU), can be used as aunit for processing an image.

The CU is an image processing unit on which coding/decoding areperformed. The CU can include information used to code or decode acoding block, that is, a block unit set of luminance samples orchrominance samples on which coding/decoding are performed, and thesamples of the coding block.

The PU is an image processing unit on which prediction is performed. ThePU can include information used to predict a prediction block, that is,a block unit set of luminance samples or chrominance samples on whichprediction is performed, and the samples of the prediction block. Here,a coding block can be classified into a plurality of prediction blocks.

The TU is an image processing unit on which transform is performed. TheTU can include information used to transform a transform block, that is,a block unit set of luminance samples or chrominance samples on whichtransform is performed, and the samples of the transform block. Here, acoding block can be classified into a plurality of transform blocks.

In an embodiment of the present invention, a block and a unit can beinterpreted as having the same meaning unless described otherwisehereinafter.

Furthermore, a current block can designate a block on which specificimage processing is being performed, such as a prediction block on whichprediction is now performed or a coding block on which prediction is nowperformed. For example, if one coding block is partitioned into twoprediction blocks, a block on which prediction is now performed, fromamong the partitioned prediction blocks, can be designated as a currentblock.

In an embodiment of the present invention, an image coding method and animage decoding method to be described later can be performed by theelements of the image coder and image decoder described with referenceto FIGS. 1 and 2. The element can include not only a hardware meaning,but also a software processing unit that can be performed by analgorithm.

Hereinafter, a method of setting the reference picture index of atemporal merging candidate disclosed in an embodiment of the presentinvention can be used both in SKIP mode in an image processing methodand merge mode, that is, one of modes, in an inter-frame predictionmethod. SKIP mode is an image processing method of outputting a block,predicted based on motion prediction information derived fromsurrounding blocks, as a reconstructed block without generating aresidual block. Merge mode, that is, one of modes, in an inter-frameprediction method is an image processing method which is the same asSKIP mode in that a block is predicted based on motion predictioninformation derived from surrounding blocks, but is different from SKIPmode in that a block reconstructed by adding a residual block and aprediction block by coding and decoding information on the residualblock is outputted. Intra-loop filtering methods, such as deblockingfiltering and a sample adaptive offset, can be additionally applied tothe outputted reconstructed block.

FIG. 3 is a conceptual diagram illustrating an inter-frame predictionmethod using merge mode in accordance with an embodiment of the presentinvention.

Referring to FIG. 3, the inter-frame prediction using merge mode can beperformed as follows.

The inter-frame prediction method using merge mode refers to a method ofderiving a merging candidate from a block neighboring a current blockand performing inter-frame prediction by using the derived mergingcandidate. The neighboring block used to derive the merging candidatecan be partitioned into a block which is located in the same picture asa current block and neighbors the current block and a block which islocated in a picture different from a picture including a current blockand at a location collocated with the current block.

Hereinafter, in an embodiment of the present invention, from amongneighboring blocks used to derive a merging candidate, a block which islocated in the same picture as a current block and neighbors the currentblock is defined as a spatial merging candidate block, and motionprediction-related information derived from the spatial mergingcandidate block is defined as a spatial merging candidate. Furthermore,from among neighboring blocks used to derive a merging candidate, ablock which is located in a picture different from a picture including acurrent block and at a location collocated with the current block isdefined as a temporal merging candidate block, and motionprediction-related information derived from the temporal mergingcandidate block is defined as a temporal merging candidate

That is, the inter-frame prediction method using merge mode is aninter-frame prediction method for predicting a current block by usingmotion prediction-related information (i.e., a spatial mergingcandidate) on a spatial merging candidate block or motionprediction-related information (i.e., a temporal merging candidate) on atemporal merging candidate block to be described later.

For example, motion vectors mvL0/L1, reference picture indicesrefIdxL0/L1, and pieces of reference picture list utilizationinformation predFlagL0/L1 can be used as the motion prediction-relatedinformation.

FIG. 3(A) shows the motion vectors mvL0/L1, the reference pictureindices refIdxL0/L1, and the pieces of reference picture listutilization information predFlagL0/L1.

A motion vector 304 is directional information and can be used for aprediction block to derive information on a pixel, located at a specificlocation, from a reference picture in performing inter-frame prediction.If inter-frame prediction is performed using a plurality of pieces ofdirectional information in a prediction block, motion vectors forrespective directions can be indicated by mvL0/L1.

A reference picture index 306 is information on the index of a pictureto which a prediction block refers in performing inter-frame prediction.If inter-frame prediction is performed using a plurality of referencepictures, reference pictures can be indexed using respective referencepicture indices refIdxL0 and refIdxL1.

The reference picture list utilization information can indicate that areference picture has been derived from what reference picture list 0308. For example, pictures i, j, and k can be stored in a referencepicture list 0 308 and used. If there are two lists in which a referencepicture is stored, information on that the reference picture has beenderived from what reference picture list can be indicated by predFlagL0and predFlagL1.

In order to perform the inter-frame prediction method using merge mode,first, a spatial merging candidate can be obtained through the followingstep (1). FIG. 3(B) discloses a spatial merging candidate and a temporalmerging candidate.

(1) A Spatial Merging Candidate is Derived from Neighboring Blocks for aCurrent block (i.e., a target prediction block).

As described above, a spatial merging candidate is motionprediction-related information derived from a spatial merging candidateblock. The spatial merging candidate block can be derived on the basisof the location of a current block.

Referring to FIG. 3(B), the existing spatial merging candidate blocks300, 310, 320, 330, and 340 have been derived based on a targetprediction block. It is assumed that the location of a pixel present atan upper left end of the target prediction block is (xP, yP), the widthof a prediction block is nPbW, the height of the target prediction blockis nPbH, and MinPbSize is the smallest size of the prediction block. Inan embodiment of the present invention hereinafter, the spatial mergingcandidate blocks of the prediction block can include a block including apixel present at (xP−1, yP+nPbH), that is, a first block (or an A0block) 300 on the left side, a block including a pixel present at (xP−1,yP+nPbH−1), that is, a second block (or an A1 block) 310 on the leftside, a block including a pixel present at (xP+nPbW, yP−1), that is, afirst block (or a B0 block) 320 at the upper end, a block including apixel present at (xP+nPbW−1, yP−1), that is, a second block (or a B1block) 330 at the upper end, and a block including a pixel present at(xP−1, yP−1), that is, a third block (or a B2 block) 340 at the upperend. Another value, for example, “MinPbSize” may be used instead of 1.In this case, a block at the same location can be indicated. Coordinatesused to indicate the block at the specific location are arbitrary, andthe block at the same location may be indicated by various otherrepresentation methods.

The locations of the spatial merging candidate blocks 300, 310, 320,330, and 340 and the number thereof and the locations of the temporalmerging candidate blocks 360 and 370 and the number thereof disclosed inFIG. 3 are illustrative, and the locations of spatial merging candidateblocks and the number thereof and the locations of temporal mergingcandidate blocks and the number thereof can be changed if they fallwithin the essence of the present invention. Furthermore, order ofmerging candidate blocks preferentially scanned when a merging candidatelist is configured may be changed. That is, the locations of candidateprediction blocks, the number thereof, and a scan order thereof, and acandidate prediction group used when a candidate prediction motionvector list is configured, described in the following embodiment of thepresent invention, are only illustrative and can be change if they fallwithin the essence of the present invention.

A spatial merging candidate can be derived from an available spatialmerging candidate block by determining whether the spatial mergingcandidate blocks 300, 310, 320, 330, and 340 are available or not.Information indicating whether a spatial merging candidate can bederived from a spatial merging candidate block or not is availabilityinformation. For example, if a spatial merging candidate block islocated outside a slice, tile, or a picture to which a current blockbelongs or is a block on which intra-frame prediction has beenperformed, a spatial merging candidate, that is, motionprediction-related information, cannot be derived from the correspondingblock. In this case, the spatial merging candidate block can bedetermined to be not available. In order to determine availabilityinformation on the spatial merging candidate, some determination methodscan be used and embodiments thereof are described in detail later.

If a spatial merging candidate block is available, motionprediction-related information can be derived and used to performinter-frame prediction using merge mode on a current block.

One coding block can be partitioned into one or more prediction blocks.That is, a coding block can include one or more prediction blocks. If aplurality of prediction blocks is included in a coding block, each ofthe prediction blocks can be indicated by specific index information.For example, if one coding block is partitioned into two predictionblocks, the two prediction blocks can be indicated by setting thepartition index of one prediction block to 0 and the partition index ofthe other prediction block to 1. If a partition index is 0, a predictionblock may be defined as another term, such as a first prediction block.If a partition index is 1, a prediction block may be defined as anotherterm, such as a second prediction block. If one coding block is furtherpartitioned into additional prediction blocks, index values indicativeof the prediction blocks can be increased. The terms defined todesignate the prediction blocks are arbitrary, and the terms may bedifferently used or differently interpreted. The partition index of aprediction block may also be used as information indicative of orderthat image processing, such as coding and decoding, is performed when aprediction block performs the image processing.

If a plurality of prediction blocks is present within one coding block,there may be a case where coding or decoding on another prediction blockmust be first performed when a spatial merging candidate for theprediction block is derived. In accordance with an embodiment of thepresent invention, a method of deriving spatial merging candidates andtemporal merging candidates in parallel to each of prediction blocksincluded in one coding block when generating a merging candidate list isadditionally disclosed in detail.

(2) Determine the Reference Picture Index of a Temporal MergingCandidate.

A temporal merging candidate is motion prediction-related informationderived from a temporal merging candidate block that is present at apicture different from a picture including a current block. The temporalmerging candidate block is derived based on a block that is at alocation collocated based on the location of the current block. The term‘colocated block’ can be used as the same meaning as the temporalmerging candidate block.

Referring back to FIG. 3, the temporal merging candidate blocks 360 and370 can include the block 360 including a pixel at a location (xP+nPSW,yP+nPSH) in the colocated picture of a current prediction block or theblock 370 including a pixel at a location (xP+(nPSW>>1), yP+(nPSH>>1))if the block 360 including the pixel at the location (xP+nPSW, yP+nPSH)is not available, on the basis of the pixel location (xP, yP) within thepicture including the prediction block. The prediction block 360including the pixel at the location (xP+nPSW, yP+nPSH) in the colocatedpicture can be called a first temporal merging candidate block (or afirst colocated block) 360, and the prediction block including the pixelat the location (xP+(nPSW>>1), yP+(nPSH>>1)) in the colocated picturecan be called a second temporal merging candidate block 370.

Finally, the final temporal merging candidate block used to derive atemporal merging candidate (or motion prediction-related information)can be at a location partially moved on the basis of the locations ofthe first temporal merging candidate block 360 and the second temporalmerging candidate block 370. For example, if only pieces of motionprediction-related information on some prediction blocks present in acolocated picture are stored in memory, a block at a location partiallymoved on the basis of the locations of the first temporal mergingcandidate block 360 and the second temporal merging candidate block 370can be used as the final temporal merging candidate block for derivingthe final motion prediction-related information. Like in a spatialmerging candidate block, the location of a temporal merging candidateblock can be changed or added unlike in FIG. 3, and an embodimentthereof is described later.

The reference picture index of a temporal merging candidate isinformation indicative of a picture that is referred in order for acurrent block to perform inter-frame predict on the basis of a motionvector mvLXCol derived from a temporal merging candidate.

FIG. 4 is a conceptual diagram illustrating inter-frame prediction usinga temporal merging candidate and the reference picture index of thetemporal merging candidate in accordance with an embodiment of thepresent invention.

Referring to FIG. 4, a current block 400, a picture 410 including thecurrent block, a temporal merging candidate block (or a colocated block)420, and a colocated picture 430 including the colocated block can bedefined.

From a viewpoint of the temporal merging candidate block 420, there is apicture 440 used in inter-frame prediction by the temporal mergingcandidate block in order to perform the inter-frame prediction on thetemporal merging candidate block 420. This picture is defined as thereference picture 440 of the colocated picture 430. Furthermore, amotion vector that is used by the temporal merging candidate block 420in order to perform inter-frame prediction from the reference picture440 of the colocated picture 430 can be defined as mvCol 470.

From a standpoint of the current block 400, a reference picture 460 usedin the inter-frame prediction of the current block 400 on the basis ofthe calculated mvCol 470 has to be defined. The reference picturedefined to be used in the inter-frame prediction of the current block400 can be called the reference picture 460 of a temporal mergingcandidate. That is, the index of the reference picture 460 of thetemporal merging candidate (i.e., the reference index of the temporalmerging candidate) is a value indicative of a reference picture used inthe temporal motion prediction of the current block 400. At the step(2), the reference picture index of a temporal merging candidate can bedetermined.

A mvCol 470, that is, a motion vector derived from the temporal mergingcandidate block 420, can be scaled and transformed into a differentvalue depending on the distance between the colocated picture 430 andthe reference picture 440 of the colocated picture and the distancebetween the picture 410 including the current block and the referencepicture 460 of the temporal merging candidate derived through the step(2).

That is, inter-frame prediction according to the temporal mergingcandidate of the current block 400 can be performed based on mvLXCol 480derived through a step (3) to be described later, on the basis of thereference picture index 460 of the temporal merging candidate derivedthrough the step (2) and the reference picture index 460 of the temporalmerging candidate. mvLXCol can be defined as a temporal motion vector.

In the existing image coding/decoding methods, the reference pictureindex of a temporal merging candidate can be determined based on thereference picture index candidate of a temporal merging candidatederived from the reference picture index of a spatial merging candidatein a target prediction block. If this method is used, there may be acase where the reference picture index of a spatial merging candidatethat has not yet been coded or decoded must be derived. In this case,the reference picture index of the spatial merging candidate can bederived only when coding or decoding on a prediction block including thecorresponding spatial merging candidate is finished. Accordingly, if thereference picture index of a temporal merging candidate is determinedbased on the reference picture index candidates of temporal mergingcandidates derived from all spatial merging candidate blocks, a processof deriving the reference pictures of the temporal merging candidatesfor a current block cannot be performed in parallel. FIG. 5 disclosesthis problem.

FIG. 5 is a conceptual diagram illustrating a case where one codingblock is partitioned into two prediction blocks.

Referring to FIG. 5, one coding block is partitioned into a firstprediction block 500 and a second prediction block 520 having an N×2Nform. Spatial merging candidate blocks for the first prediction block500 are derived on the basis of the location of the first predictionblock 500 as in FIG. 5(A), and spatial merging candidate blocks for thesecond prediction block 520 are derived on the basis of the location ofthe second prediction block 520 as in FIG. 5(B). Although not shown, intemporal merging candidate blocks, temporal merging candidates can bederived on the basis of the location of each of prediction blocks.

The spatial merging candidate blocks of the first prediction block 500are outside the first prediction block 500 and are at locations includedin blocks on which coding or decoding has already been performed.

In contrast, an A1 block 530, from among the spatial merging candidateblocks of the second prediction block 520, is present within the firstprediction block 500. Accordingly, after prediction on the firstprediction block 500 is performed, motion prediction-related information(e.g., a motion vector, a reference picture index, and reference picturelist utilization information) on the A1 block 530 can be known.Furthermore, the motion prediction-related information of the A0 block550 cannot be derived because the A0 block 550 is at a location that hasnot yet been coded or decoded.

If the reference picture index of a temporal merging candidate isderived from the motion prediction-related information of the A1 block530, it can be derived after coding and decoding on the first predictionblock 500 are finished. Furthermore, the reference picture index cannotbe derived from the A0 block 550. That is, since the reference pictureindices of some spatial merging candidate blocks cannot be derived, thereference picture indices of temporal merging candidates for respectiveprediction blocks cannot be derived in parallel.

In an embodiment of the present invention, in order to solve theproblem, methods of deriving the reference picture indices of temporalmerging candidates (or the reference indices of temporal mergingcandidates) for prediction blocks are disclosed.

If a method of deriving the reference picture indices of temporalmerging candidates in accordance with an embodiment of the presentinvention is used, processes of deriving the reference picture indicesof temporal merging candidates for some prediction blocks can beperformed in parallel. Since the reference picture indices of temporalmerging candidates are derived in parallel, inter-frame predictionprocesses using merge mode for a plurality of prediction blocks includedin one coding block can be performed in parallel.

Hereinafter, in an embodiment of the present invention, a method ofderiving the reference picture index of a temporal merging candidate isdisclosed and additionally described in detail.

(3) Derive Motion Prediction-Related Information on a Temporal MergingCandidate Block.

At the step (3), in order to perform motion prediction based on atemporal merging candidate, temporal merging candidates, such asinformation on whether a temporal merging candidate block is availableor not (availableFlagCol), reference picture list utilizationinformation (PredFlagLXCol), and information on the motion vector(mvLXCol) of a temporal merging candidate, can be derived. The motionprediction-related information derived from the temporal mergingcandidate can be defined as a term ‘temporal merging candidate’. Theavailability information on the temporal merging candidate blockindicates whether a temporal merging candidate can be derived from thetemporal merging candidate block or not. The temporal merging candidatecan be included in a merging candidate list on the basis of theavailability information on the temporal merging candidate block.

(4) Derive a Merging Candidate List.

A merging candidate list can include information on a merging candidatethat can be used in inter-frame prediction using merge mode on the basisof availability information on a merging candidate block (i.e., aspatial merging candidate block or a temporal merging candidate block).One of merging candidates included in a merging candidate list can beused to perform inter-frame prediction using merge mode on a currentblock. Information on whether what merging candidate will be used topredict a current block (i.e., a merging index) can be coded in a codingstep and transmitted to a decoder.

A merging candidate list can be generated with the following order ofpriority.

1) If an A1 block is available, a merging candidate derived from the A1block

2) If a B1 block is available, a merging candidate derived from the B1block

3) If a B0 block is available, a merging candidate derived from the B0block

4) If an A0 block is available, a merging candidate derived from the A0block

5) If a B2 block is available, a merging candidate derived from the B2block

6) If a Col block is available, a merging candidate derived from the Colblock

The merging candidate list can include, for example, 0 to 5 mergingcandidates depending on the number of available blocks. If the number ofblocks used to derive a merging candidate is many, more mergingcandidates may be included in the merging candidate list.

(5) If the number of merging candidates included in a merging candidatelist is smaller than a maximum number of merging candidates that can beincluded in the merging candidate list, an additional merging candidateis derived.

An additional merging candidate can be a candidate generated bycombining pieces of motion prediction-related information on theexisting merging candidates (i.e., a combined merging candidate) or canbe a 0-vector merging candidate (i.e., a zero merging candidate) Here,the 0-vector merging candidate designates a merging candidate having amotion vector (0,0).

(6) Determine a Merging Candidate Applied to Inter-Frame PredictionPerformed on a Current Block, from Among Merging Candidates Included ina Merging Candidate List, and Set Motion Prediction-Related Informationon the Determined Merging Candidate as Motion Prediction-RelatedInformation on a Current Block.

In a decoding process, inter-frame prediction using merge mode can beperformed on a current block on the basis of a merging indexmerge_idx[xP][yP], that is, information on which one of candidatesincluded in a merging candidate list is used in inter-frame predictionperformed on the current block.

Through a procedure of the step (1) to the step (6), motionprediction-related information on a current block can be derived andinter-frame prediction can be performed on the current block based onthe derived motion prediction-related information.

An embodiment of the present invention discloses a method of derivingthe reference picture indices of temporal merging candidates for aplurality of prediction blocks, included in one coding block, inparallel in setting the reference picture index of a temporal mergingcandidate at the step (2) is disclosed.

Various kinds of methods below can be used as the method of deriving thereference picture indices of temporal merging candidates for a pluralityof prediction blocks, included in a coding block, in parallel.

1) A method of setting the location of a spatial merging candidateblock, used to derive the reference picture index candidate of atemporal merging candidate for a target prediction block (i.e., acurrent block), as a location at which a coding block including thecurrent block is located and on which coding or decoding has alreadybeen performed.

2) A method of, if the location of a spatial merging candidate blockused to derive the reference picture index candidate of a temporalmerging candidate for a target prediction block (i.e., a current block)is within a coding block or a location on which coding has not yet beenperformed, setting the reference picture index candidate of a temporalmerging candidate derived from a spatial merging candidate at thecorresponding location to ‘0’.

3) A method of setting the reference picture index of the temporalmerging candidate of a target prediction block (i.e., a current block)to ‘0’ that is a fixed value.

4) A method of, if the location of a spatial merging candidate blockreferred to derive the reference picture index candidate of the temporalmerging candidate of a target prediction block (i.e., e current block)is within a coding block or a location on which coding has not yet beenperformed, not using the reference picture index of the spatial mergingcandidate block at the corresponding location in order to derive thereference picture index of the temporal merging candidate.

5) A method of previously determining a spatial merging candidate blockat a specific location that is referred to derive the reference pictureindex of the temporal merging candidate of a target prediction block(i.e., a current block) and deriving the reference picture index of thetemporal merging candidate from the spatial merging candidate block atthe specific location.

6) A method of, if the locations of some of the spatial mergingcandidate blocks of spatial merging candidates derived to performmergence on a target prediction block (i.e., a current block) are withina coding block or locations on which coding has not yet been performedand thus pieces of information on the reference picture indices oftemporal merging candidates cannot be derived from the spatial mergingcandidate blocks at the corresponding locations, fixing the spatialmerging candidate blocks at the corresponding locations as locationsoutside the coding block on which coding or decoding has been performed.

The following embodiments of the present invention disclose the methodsof deriving the reference picture index of a temporal merging candidatein detail.

First, problems occurring when determining the reference picture indexof a temporal merging candidate in the prior art, described withreference to FIG. 5, are described in detail with reference to FIG. 6.

FIG. 6 is a conceptual diagram illustrating a method of setting thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

Referring to FIG. 6, one coding block (e.g., a 2N×2N form) can bepartitioned into two prediction blocks (e.g., N×2N). In the firstprediction block 600 of the two partitioned blocks, all spatial mergingcandidate blocks 605, 610, 615, 620, and 625 are present outside thecoding block. In contrast, in the second prediction block 650 of the twopartitioned blocks, some (e.g., 655, 665, 670, and 675) of spatialmerging candidate blocks 655, 660, 665, 670, and 675 are present outsidethe coding block, and some (e.g., 660) of the spatial merging candidateblocks 655, 660, 665, 670, and 675 are present within the coding block.

The reference picture index of a temporal merging candidate for acurrent block (i.e., a target prediction block) can be derived from thereference picture index of a spatial merging candidate. That is, thereference picture index of a temporal merging candidate for a currentblock can be derived based on information on a reference picture indexthat has been used by a spatial merging candidate block to performinter-frame prediction.

For example, it can be assumed that the reference picture indices ofthree of a plurality of spatial merging candidates for a current blockare refIdxLXA, refIdxLXB, and refIdxLXC. Pieces of information on thereference picture indices refIdxLXA, refIdxLXB, and refIdxLXC can becomethe reference picture index candidates of temporal merging candidates,and the reference picture index values of the temporal mergingcandidates can be derived based on the reference picture indexcandidates of the temporal merging candidates.

If the above method is used, spatial merging candidate blocks for acurrent block need to be coded or decoded in advance because pieces ofinformation on the reference picture indices of the spatial mergingcandidate blocks for the current block are necessary to derive thereference picture indices of temporal merging candidates for the currentblock.

Referring back to FIG. 6, the first prediction block 600 is a block inwhich the spatial merging candidates are included in locations outsidethe coding block on which coding or decoding has already been performedas described above. Accordingly, if the first prediction block 600 is acurrent block on which prediction is performed, the reference pictureindex candidates of temporal merging candidates for the first predictionblock 600 can be directly derived from the spatial merging candidateblocks of the first prediction block 600.

In the second prediction block 650, however, some (e.g., 660) of thespatial merging candidates are present in the first prediction block 600that is within the coding block as described above. Accordingly, wheninter-frame prediction using merge mode is performed on the secondprediction block 650, the reference picture indices of temporal mergingcandidates for the first prediction block 650 cannot be derived untilthe A1 block 660 is coded or decoded, that is, until prediction isperformed on the first prediction block 600 including the A1 block 660.In this case, there is a problem in that inter-frame prediction usingmerge mode cannot be performed on the first prediction block 600 and thesecond prediction block 650 in parallel because the temporal mergingcandidates of the second prediction block 650 are not derived untilprediction is performed on the first prediction block 600. In order tosolve the problem, a variety of methods can be used.

Only some of the partition forms of a prediction block are disclosed inthe following embodiments of the present invention, for convenience ofdescription, but the present invention can be applied to the partitionforms of several prediction blocks of a coding block and embodimentsthereof are also included in the scope of the present invention.

FIG. 7 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

The embodiment of FIG. 7 discloses a method of setting the locations ofspatial merging candidate blocks to which reference is made by aprediction block in order to derive the reference picture indices oftemporal merging candidates as locations outside a coding blockincluding a current prediction block.

FIG. 7(A) shows a case where one coding block is partitioned into twoprediction blocks 700 and 750 having an N×2N form.

All the spatial merging candidate blocks of the first prediction block700 are at locations outside a coding unit on which coding or decodinghas already been performed. Thus, the reference picture index candidatesof temporal merging candidates for the first prediction block 700 can bedirectly derived by using the already coded or decoded spatial mergingcandidate blocks.

In the case of the second prediction block 750, however, the locationsof some (e.g., 710 and 720) of spatial merging candidate blocks used toderive the reference picture indices of temporal merging candidates canbe changed, and the reference picture indices of the temporal mergingcandidates can be derived from the changed locations.

In order to derive the reference picture index candidates of thetemporal merging candidates, the spatial merging candidate block 710 canbe replaced with a block 715 outside the coding block without using thespatial merging candidate block 710 included in the coding unit, fromamong the spatial merging candidate blocks of the second predictionblock 750, and the reference picture index of the block 715 can be usedas the reference picture index candidate of a temporal mergingcandidate.

Furthermore, the spatial merging candidate block 720 can be replacedwith a block 725 outside the coding block without using the block 720outside the coding unit on which coding or decoding has not yet beenperformed, from among the spatial merging candidate blocks, and thereference picture index of the block 725 can be used as the referencepicture index candidate of a temporal merging candidate.

That is, the reference picture index candidates of the temporal mergingcandidates can be derived by using an A0′ block 725 and an A1′ block 715outside the coding block instead of the A0 block 710 and the A1 block720 of the second prediction block 750.

If the above method is used, all the spatial merging candidate blocksused to derive the reference picture indices of the temporal mergingcandidates can become blocks included in an already coded block in thesecond prediction block 750. Accordingly, in the second prediction block750, the reference picture indices of the temporal merging candidatescan be derived irrespective of whether or not a prediction process hasbeen performed on the first prediction block 700.

FIG. 7(B) shows a case where one coding block is partitioned into twoprediction blocks having a 2N×N form.

As in FIG. 7(A), in FIG. 7(B), instead of a B1 block 780, that is, ablock included within the coding block, and a B0 block 790, that is, ablock on which coding or decoding has not yet been performed, from amongthe spatial merging candidate blocks of a second prediction block 770, aB1′ block 785 and a B0′ block 795 that are already coded blocks can beused to derive the reference picture indices of temporal mergingcandidates for the second prediction block 770.

FIG. 8 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

The embodiment of FIG. 8 discloses a method of setting the referencepicture index candidates of temporal merging candidates, derived from aspatial merging candidate block present within a coding block and aspatial merging candidate block present in a location on which coding ordecoding has not yet been performed, to ‘0’, if the locations of spatialmerging candidate blocks referred to derive the reference pictureindices of temporal merging candidates for a target prediction block(i.e., a current block) are within the coding block including thecurrent block or are locations on which coding or decoding has not yetbeen performed.

FIG. 8(A) shows a case where one coding block is partitioned into twoprediction blocks having an N×2N form.

Referring to FIG. 8(A), all the spatial merging candidate blocks of afirst prediction block 800 are at locations outside the coding unit onwhich coding or decoding has already been performed. Accordingly, thereference picture index candidates of temporal merging candidates forthe first prediction block 800 can be derived from the spatial mergingcandidate blocks of the first prediction block 800.

In the case of a second prediction block 850, assuming that thereference picture indices of some spatial merging candidate blocks(e.g., 810 and 820) are ‘0’, the reference picture index candidates oftemporal merging candidate for the second prediction block 850 can bederived. In relation to a spatial merging candidate block located withinthe coding block including a target prediction block (i.e., a currentblock) or a spatial merging candidate block at a location on whichcoding or decoding has not yet been performed, the reference pictureindex candidate of a temporal merging candidate derived from acorresponding spatial merging candidate block can be set to ‘0’ and thereference picture index of a temporal merging candidate for the currentblock can be derived from the set reference picture index candidate.

For example, a process of setting the reference picture index candidatesof temporal merging candidates, derived from the A0 block 810 and the A1block 820 of the second prediction block 850, to ‘0’ in advance whenderiving the reference picture index candidates of the temporal mergingcandidates and deriving the reference picture indices of the temporalmerging candidates from the set reference picture index candidates canbe used.

FIG. 8(B) shows a case where one coding block is partitioned into twoprediction blocks having a 2N×N form.

All the spatial merging candidate blocks of a first prediction block 860are at locations outside the coding unit on which coding or decoding hasbeen completed. Accordingly, the reference picture index candidates oftemporal merging candidates for the first prediction block 860 can bedirectly derived from the spatial merging candidate blocks of the firstprediction block 860.

The reference picture index candidate of a temporal merging candidatederived from a spatial merging candidate block 880 included in aprediction block on which prediction has not yet been performed or somespatial merging candidate blocks (e.g., 890) at locations on which acoding or decoding process has not yet been performed can be set to ‘0’,when deriving the reference picture indices of temporal mergingcandidates for a second prediction block 870. The reference pictureindex candidates of the temporal merging candidates can be derived fromthe set reference picture index candidates.

For example, the above method can be used in a process of setting thereference picture indices of temporal merging candidates derived from aB0 block 880 and a B1 block 890, that is, the spatial merging candidateblocks of the second prediction block 870, to ‘0’ and deriving thereference picture indices of the temporal merging candidates for thesecond prediction block 870.

FIG. 9 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

The embodiment of FIG. 9 discloses a method in which a prediction blocksets the reference picture index of a temporal merging candidate to ‘0’,that is, a fixed value, and using the set reference picture index.

FIG. 9(A) shows a case where one coding block is partitioned into twoprediction blocks having an N×2N form.

Referring to FIG. 9(A), in order to derive the reference picture indicesof temporal merging candidates for a first prediction block 900 and asecond prediction block 950, the reference picture index values oftemporal merging candidates can be set to ‘0’ and used, without usingspatial merging candidate blocks 905 to 925 and 930 to 947. If thismethod is used, the degree of complexity in the deriving of coding anddecoding can be reduced and the speed of coding and decoding can beincreased because a step of deriving the reference picture indices oftemporal merging candidates is not performed. Furthermore, the referencepicture indices of temporal merging candidates for a current block canbe derived without a need to wait for until prediction on otherprediction blocks included in a current coding block is performed.Accordingly, the reference picture indices of temporal mergingcandidates for a plurality of prediction blocks included in one codingblock can be derived in parallel.

FIG. 9(B) shows a case where one coding block is partitioned into twoprediction blocks having a 2N×N form.

Likewise, in FIG. 9(B), in order to derive the reference picture indicesof temporal merging candidates for a first prediction block 960 and asecond prediction block 990, the reference picture index values of thetemporal merging candidates can be fixed to ‘0’ and used without usingspatial merging candidates.

In FIG. 9, ‘0’ is marked in the spatial merging candidate blocks, forconvenience of description. However, when actually deriving thereference picture indices of temporal merging candidates, a value set to‘0’ can be used without a procedure for searching for the referencepicture indices of the spatial merging candidate blocks. ‘0’ is only anexample of a fixed picture index, and another picture index other than 0may be used and embodiments thereof are also included in the scope ofthe present invention.

FIG. 10 is a conceptual diagram illustrating a method of deriving thereference picture indices of temporal merging candidates in accordancewith an embodiment of the present invention.

The embodiment of FIG. 10 discloses a method of, if the location of aspatial merging candidate block referred to derive the reference pictureindex of a temporal merging candidate for a current block (i.e., atarget prediction block) is within a coding block including the currentblock or at a location on which coding has not yet been performed, notusing the reference picture index of the spatial merging candidate blockas a candidate for deriving the reference picture index of the temporalmerging candidate.

FIG. 10(A) shows a case where one coding block is partitioned into twoprediction blocks having an N×2N form.

Referring to FIG. 10(A), the A1 block 1030 and the A0 block 1020 of asecond prediction block 1010 are a block within the coding blockincluding a current block and a block at a location on which coding ordecoding has not yet been performed. Pieces of information on thereference picture indices of the A1 block 1030 and the A0 block 1020cannot be used when deriving the reference picture indices of temporalmerging candidates for a first prediction block 1000.

Accordingly, when deriving the reference picture indices of the temporalmerging candidates from the second prediction block 1010, the pieces ofinformation on the reference picture indices of the A1 block 1030 andthe A0 block 1020 can be set to ‘−1’. If the reference picture indexvalue of a specific spatial merging candidate block is ‘−1’, the spatialmerging candidate block can indicate a block that is not used to derivethe reference picture index of a temporal merging candidate.

FIG. 10(B) shows a case where one coding block is partitioned into twoprediction blocks having a 2N×N form.

Referring to FIG. 10(B), the B1 block 1060 of a second prediction block1050 is a spatial merging candidate block within the coding block and isa block whose reference picture index information can be known only whenprediction is performed on a first prediction block 1040. The B0 block1070 of the second prediction block 1050 is a spatial merging candidateblock at a location on which coding has not yet been performed, andinformation on the reference picture index thereof cannot be known.

In this case, in order to derive the reference picture indices oftemporal merging candidates from the first prediction block 1040 and thesecond prediction block 1050 in parallel, pieces of information on thereference picture indices of the B1 block 1060 and the B0 block 1070 canbe set to ‘−1’. That is, the B0 block 1070 and the B1 block 1060 may notbe used as blocks for deriving the reference picture index candidates oftemporal merging candidates for the second prediction block 1050.

FIG. 11 is a conceptual diagram illustrating a method of deriving thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

The embodiment of FIG. 11 discloses a method of previously determining aspecific spatial merging candidate block referred by a prediction blockin order to derive the reference picture index of a temporal mergingcandidate and deriving the reference picture index of the temporalmerging candidate from the specific spatial merging candidate block.

FIG. 11(A) shows a case where one coding block is partitioned into twoprediction blocks having an N×2N form.

A first prediction block 1100 and a second prediction block 1120 canshare spatial merging candidate blocks A0, A1, B0, B1, and B2. That is,the spatial merging candidate blocks A0, A1, B0, B1, and B2 used toperform inter-frame prediction using merge mode in the first predictionblock 1100 and the second prediction block 1120 can be blocks outsidethe coding block.

A reference picture index for the temporal merging of the firstprediction block 1100 can be set to the reference picture index value ofthe B1 block 1105. That is, the fixed reference picture index of aspatial merging candidate block at a specific location of a predictionblock can be set to a reference picture index value for the temporalmerging of a current block depending on a partition form.

If the B1 block 1125 is not available, the reference picture index valuecan be set to ‘0’ and used.

Like in the second prediction block 1120, the reference picture indexvalue of the A1 block 1125 can be used as a reference picture index fortemporal merging. If the B1 block 1105 is not available, the referencepicture index value can be set to ‘0’ and used.

FIG. 11(B) shows a case where one coding block is partitioned into twoprediction blocks having a 2N×N form.

A first prediction block 1150 and a second prediction block 1170 canshare spatial merging candidate blocks A0, A1, B0, B1, and B2. That is,spatial merging candidate blocks for performing inter-frame predictionusing merge mode in the first prediction block 1150 and the secondprediction block 1170 can be blocks outside the coding block.

A reference picture index for the temporal merging of the firstprediction block 1150 can be set to the reference picture index value ofthe A1 block 1155. That is, the reference picture index of a spatialmerging candidate block at a specific location of a prediction block canbe set to a reference picture index value for the temporal merging of acurrent block depending on a partition form.

If the B1 block 1175 is not available, the reference picture index valuecan be set to ‘0’ and used.

Like in the second prediction block 1170, the reference picture indexvalue of the B1 block 1175 can be used as a reference picture index fortemporal merging. If the B1 block 1175 is not available, the referencepicture index value can be set to ‘0’ and used.

FIG. 12 is a conceptual diagram illustrating a method of deriving thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

The embodiment of FIG. 12 discloses a method of previously determining aspecific spatial merging candidate block referred by a target predictionblock in order to derive the reference picture index of a temporalmerging candidate and deriving the reference picture index of thetemporal merging candidate from the specific spatial merging candidateblock.

Referring to FIG. 12, different spatial merging candidate blocks can beused to derive the reference picture index of a temporal mergingcandidate depending on a form of a prediction block partitioned from onecoding block.

For example, in a prediction block, one of an A1 block and a B1 block,from among spatial merging candidate blocks, can be used as a block forderiving the reference picture index of a temporal merging candidate.From among the two spatial merging candidate blocks, a spatial mergingcandidate block within a coding block is not used to derive thereference picture index of the temporal merging candidate, but a spatialmerging candidate block outside the coding block can be used to derivethe reference picture index of the temporal merging candidate.

Although FIG. 12(A) showing a case where a coding block is partitionedinto prediction blocks having an N×2N form and FIG. 12(B) showing a casewhere a coding block is partitioned into prediction blocks having a 2N×Nform are illustrated for convenience of description, the same method canbe applied to a coding block partitioned in various forms.

FIG. 12(A) shows a case where a coding block is partitioned intoprediction blocks having an N×2N form.

If one coding block is partitioned into prediction blocks having an N×2Nform, the reference picture index of a B1 block 1220, that is, a spatialmerging candidate located outside the coding block and at a location onwhich coding or decoding has already been performed, from among twospatial merging candidate blocks (e.g., an A1 block 1200 and the B1block 1220), can be set as the reference picture index of a temporalmerging candidate for a second prediction block 1210.

FIG. 12(B) shows a case where a coding block is partitioned intoprediction blocks having a 2N×N size.

If one coding block is partitioned into prediction blocks having a 2N×Nform, the reference picture index of an A1 block 1240, that is, aspatial merging candidate outside the coding block, from among twospatial merging candidate blocks (e.g., the A1 block 1240 and a B1 block1260), can be set as the reference picture index of a temporal mergingcandidate for a second prediction block 1250.

FIG. 13 is a conceptual diagram illustrating a method of deriving thereference picture index of a temporal merging candidate in accordancewith an embodiment of the present invention.

The embodiment of FIG. 13 discloses a method of, if the locations ofsome of the spatial merging candidates of a prediction block are withina coding block or placed at locations on which coding has not yet beenperformed, fixing the locations of the spatial merging candidate blocksof the corresponding prediction block to locations outside the codingblock and using the fixed locations.

FIG. 13(A) shows a case where one coding block is partitioned into twoprediction blocks having an N×2N form.

A first prediction block 1300 can determine spatial merging candidateblocks 1305, 1310, 1315, 1320, and 1325 on the basis of the firstprediction block 1300. In contrast, a second prediction block 1330 canfix spatial merging candidate blocks to blocks 1335, 1340, 1345, 1350,and 1355 placed at locations outside the coding block and use the fixedspatial merging candidate blocks. That is, the spatial merging candidateblocks 1335, 1340, 1345, 1350, and 1355 can be derived on the basis ofthe coding block, and the derived spatial merging candidate blocks 1335,1340, 1345, 1350, and 1355 can be used in inter-frame prediction usingmerge mode for the second prediction block 1330.

FIG. 13(B) shows a case where one coding block is partitioned into twoprediction blocks having a 2N×N form.

Likewise, in FIG. 13(B), a first prediction block can use spatialmerging candidate blocks derived on the basis of a prediction block. Incontrast, the spatial merging candidate blocks 1365, 1370, 1375, 1380,and 1385 of a second prediction block 1360 can be derived on the basisof the coding block.

FIG. 14 is a flowchart illustrating a method of including a temporalmerging candidate in a merging candidate list in accordance with anembodiment of the present invention.

The embodiment of FIG. 14 discloses a process of deriving the referencepicture index of a temporal merging candidate by using an index valuecalculated by the above-described method of deriving the referencepicture index of a temporal merging candidate and of including thetemporal merging candidate in a merging candidate list.

Referring to FIG. 14, the reference picture index of a temporal mergingcandidate is derived at step S1400.

The reference picture index of the temporal merging candidate refers tothe reference picture index of a picture referred by a current block(i.e., a target prediction block) in order to perform inter-frameprediction using merge mode as described above. The reference pictureindex of the temporal merging candidate can be derived by severalmethods of deriving the reference picture indices of temporal mergingcandidates in parallel in relation to a prediction block. For example,the reference picture index of the temporal merging candidate can bederived by several methods, such as 1) a method of always placing thespatial location of a spatial merging candidate block to be referredoutside a coding block, 2) a method of replacing a reference pictureindex value, derived from a spatial merging candidate block to bereferred, with ‘0’ if the spatial location of the spatial mergingcandidate block is within a coding block, and 3) a method of fixing thereference picture index of a temporal merging candidate to ‘0’unconditionally.

A temporal merging candidate is derived at step S1410.

As described above, the temporal merging candidate can be motionprediction-related information (e.g., predFlag or mvLXCol) derived froma prediction block (e.g., a first temporal merging candidate block)which includes a pixel at a location (xP+nPbW, yP+nPbH) in the colocatedpicture of a current block on the basis of the location (xP, yP) of apixel within a picture including the prediction block. If the predictionblock including the pixel at the location (xP+nPbW, yP+nPbH) in thecolocated picture is not available or is a block predicted by anintra-frame prediction method, motion prediction-related information(e.g., a temporal merging candidate) can be derived from a predictionblock (e.g., a second temporal merging candidate block) including apixel at a location (xP+(nPbW>>1), yP+(nPbH>>1)).

Finally, a final temporal merging candidate block (i.e., a colocatedblock) used to derive the motion prediction-related information can be ablock at a location that has been partially moved on the basis of thelocations of the first temporal merging candidate block and the secondtemporal merging candidate block. For example, if only pieces of motionprediction-related information on some blocks are stored in memory, atemporal merging candidate block present at a location partially movedon the basis of the locations of the first temporal merging candidateblock and the second temporal merging candidate block can be determinedas the final colocated block for deriving the temporal merging candidate(i.e., motion prediction-related information).

In deriving the temporal merging candidate, different temporal mergingcandidates can be derived depending on whether a current block is ablock using a single merging candidate list or a block not using asingle merging candidate list. If the current block is a block using asingle merging candidate list, a plurality of prediction blocks includedin a coding block can use temporal merging candidates derived from onetemporal merging candidate block. If the current block is a block notusing a single merging candidate list, a merging candidate list for aplurality of prediction blocks included in a coding block can begenerated and inter-frame prediction using merge mode can be performedindividually. That is, in this case, the inter-frame prediction can beperformed by using temporal merging candidates derived from the temporalmerging candidate block for each prediction block. An example in whichinter-frame prediction is performed by using a single merging candidatelist is described below.

FIG. 15 is a conceptual diagram illustrating a method of generating asingle merging candidate list by sharing all spatial merging candidatesand temporal merging candidates in a plurality of prediction blocks inaccordance with an embodiment of the present invention.

The embodiment of FIG. 15 discloses a method of a plurality ofprediction blocks, partitioned from one coding block, generating asingle merging candidate list by sharing all spatial merging candidatesand temporal merging candidates determined based on the coding block.

Referring to FIG. 15, a first prediction block 1500 and a secondprediction block 1550 can derive spatial merging candidates from thesame spatial merging candidate block and share the derived spatialmerging candidates. Spatial merging candidate blocks for the firstprediction block 1500 and the second prediction block 1550 are blocksdetermined based on a coding block, and an A0 block 1505, an A1 block1510, a B0 block 1515, a B1 block 1520, and a B2 block 1525 can be usedas the spatial merging candidate blocks.

The location of each of the spatial merging candidate blocks can be alocation including a pixel shown in the drawing on the basis of theupper left location (xC, yC) and nCS (i.e., the size of the codingblock) of the coding block.

The A0 block 1505 can be a block including a pixel at a location (xC−1,yC+nCS), the A1 block 1510 can be a block including a pixel at alocation (xC−1, yC+nCS−1), the B0 block 1515 can be a block including apixel at a location (xC+nCS, yC−1), the B1 block 1520 can be a blockincluding a pixel at a location (xC+nCS−1, yC−1), and the B2 block 1525can be a block including a pixel at a location (xC−1, yC−1).

Furthermore, the first prediction block 1500 and the second predictionblock 1550 can share temporal merging candidates. Temporal mergingcandidate blocks 1560 and 1570 for deriving the temporal mergingcandidates shared by the first prediction block 1500 and the secondprediction block 1550 can be blocks at locations derived on the basis ofthe upper left locations (xC, yC) of the coding block and the size nCSof the coding block.

The reference picture indices of the temporal merging candidates can bederived by the aforementioned methods.

For example, the temporal merging candidate blocks 1560 and 1570 caninclude the prediction block 1560 including a pixel at a location(xC+nCS, yC+nCS) in the colocated picture of a current prediction blockon the basis of the pixel location (xC, yC) within the picture includingthe prediction block or can be the prediction block 1570 including apixel at a location (xC+(nCS>>1), yC+(nCS>>1)) if the prediction blockincluding the pixel at the location (xC+nCS, yC+nCS) is not available.

If temporal merging candidates are not shared, each of the temporalmerging candidates for the first prediction block 1500 and the secondprediction block 1550 can be derived.

If a method of deriving a single merging candidate list is used,inter-frame prediction can be performed by parallel merging processingperformed on each prediction block, and a merging candidate list foreach prediction block does not need to be derived separately.Accordingly, by using a single merging candidate list in accordance withan embodiment of the present invention, an image processing speed can beimproved in apparatuses, such as Ultra-High Definition TeleVision(UHDTV) that requires a large amount of data processing.

FIG. 15 discloses only the first N×2N prediction block 1500 and thesecond N×2N prediction block 1550 each partitioned in an N×2N form, butthis method can also be applied to prediction blocks partitioned invarious forms, such as blocks having different partition forms (e.g.,2N×N, 2N×nU, 2N×nD, nL×2N, nR×2N, and N×N).

Furthermore, in this method, whether or not to apply a single mergingcandidate list can be differently determined depending on the size of ablock or a partition depth. For example, information on whether a singlemerging candidate list can be used in a specific block or not can bederived on the basis of pieces of information on the size of a block andthe size of a coding block on which a merging process can be performedin parallel. For example, information on whether a single mergingcandidate list can be used in a specific block or not can be representedby flag information. A flag indicating whether or not a single mergingcandidate list can be used in a specific block can be defined assingleMCLflag (i.e., a single merge candidate list flag). For example,if the single merge candidate list flag singleMCLflag is 0, it canindicate that a block does not use a single merging candidate list. Ifthe single merge candidate list flag singleMCLflag is 1, it can indicatethat a block uses a single merging candidate list. Spatial mergingcandidates for a prediction block can be derived on the basis of acoding block based on a value of the single merge candidate list flagsingleMCLflag.

For example, the size of a block on which a merging process can beperformed in parallel can derive flag information indicating that aprediction block, partitioned from an 8×8 coding block on the basis ofinformation indicative of a value greater than a 4×4 size andinformation indicating that the size of a current block is 8×8, uses asingle merging candidate list. The derived flag can be used to derivethe spatial merging candidates and temporal merging candidates of aprediction block on the basis of a coding block.

Referring back to FIG. 14, availability information on the temporalmerging candidate and a temporal motion vector can be derived on thebasis of information on the reference picture index of the temporalmerging candidate derived to derive the temporal merging candidate atthe step S1410.

The availability information on the temporal merging candidate can beused as information indicating whether the temporal merging candidatecan be derived on the basis of a temporal merging candidate block. Thetemporal motion vector can be derived if the temporal merging candidateis available.

Referring back to FIG. 4, the temporal motion vector mvLXCol can bescaled and derived on the basis of the distance between two picturesderived based on the index of the picture 430 including a temporalmerging candidate and the index of the reference picture 440 referred bythe colocated picture 410 and the distance between pictures derivedbased on the index of the colocated picture 410 including the currentblock 400 and the index of the reference picture of a temporal mergingcandidate (i.e., the index of the reference picture 460 referred by thecurrent block 400 in inter-frame prediction).

If the temporal merging candidate is available, the temporal mergingcandidate is included in a merging candidate list at step S1420.

When configuring the merging candidate list, if the temporal mergingcandidate is available based on availability information on the temporalmerging candidate derived at the step S1410, a corresponding block canbe included in the merging candidate list.

FIG. 16 discloses a method in which prediction blocks within the samecoding block share spatial merging candidates and temporal mergingcandidates only when the size of a block is equal to or smaller than aspecific size.

FIG. 16 is a conceptual diagram illustrating a method of generating asingle candidate list in accordance with an embodiment of the presentinvention.

The embodiment of FIG. 16 discloses a method in which prediction blockswithin the same coding block share spatial merging candidates andtemporal merging candidates when the size of the coding block is equalto or smaller than a specific size in inter-frame prediction using mergemode.

Several pieces of information can be used to use a method of sharing asingle merging candidate list only in blocks that satisfy a specificcondition. For example, information on whether a current block uses asingle merging candidate list or not can be derived based on informationon the size of a block on which parallel merging processing can beperformed and information on the size of a current coding block. Spatialmerging candidates and temporal merging candidates for a predictionblock can be derived on the basis of a coding block that satisfies thespecific condition based on the pieces of derived information.

Referring to FIG. 16(A), only when conditions that the size of a blockon which parallel merging processing can be performed is 8×8 or greaterand the size of a coding block is 8×8 are satisfied, for example,prediction blocks partitioned from the coding block can share a singlemerging candidate list.

It is assumed that a first coding block CU0 1600 has a size of 32×32, asecond coding block CU1 1610 has a size of 16×16, a third coding blockCU2 1620 has a size of 32×32, a fourth coding block CU3 1630 has a sizeof 16×16, and a fifth coding block CU4 1640 has a size of 8×8.

FIG. 16(B) is a conceptual diagram only showing spatial mergingcandidate blocks for some coding blocks.

Referring to FIG. 16(B), the second coding block 1610 can be partitionedinto two prediction blocks 1615 and 1618 having an nL×2N form, and thefifth coding block 1640 can be partitioned into two prediction blocks1645 and 1650 having an N×2N form. In FIG. 16(B), it is assumed that asingle merging candidate list for only the coding block 1640 having the8×8 size is generated.

Each of the first prediction block 1615 and the second prediction block1618 of the second coding block 1610 can derive spatial mergingcandidates for each prediction block and generate a merging candidatelist for each prediction block based on the derived spatial mergingcandidates.

The size of the fifth coding block 1640 is 8×8, and the fifth codingblock 1640 can satisfy conditions of the size of a block on whichparallel merging processing can be performed and conditions of the sizeof a current coding block. In this case, the third prediction block 1645included in the fifth coding block 1640 and the fourth prediction block1650 can generate a single merging candidate list based on the spatialmerging candidates and the temporal merging candidates derived on thebasis of the location and size of a coding block. Accordingly, thereference picture index of a temporal merging candidate can be derivedas one value.

The reference picture index of the temporal merging candidate can bederived by the aforementioned methods.

The above-described image coding and image decoding methods can beimplemented in the elements of the image coder and the image decoderdescribed with reference to FIGS. 1 and 2.

Although the present invention has been described, a person havingordinary skill in the art will appreciate that the present invention maybe modified and changed in various manners without departing from thespirit and scope of the present invention which are written in theclaims below.

What is claimed is:
 1. An image decoder comprising: one or moreprocessors configured to select a collocated picture, the collocatedpicture having a different temporal order from a current picture thatincludes a current prediction block, and the collocated picturecomprising a temporal neighboring block which is used to derive atemporal merging candidate of the current prediction block, derive aspatial merging candidate by using a spatial neighboring block, derivethe temporal merging candidate by using the temporal neighboring block,generate a merging candidate list including the spatial mergingcandidate and the temporal merging candidate, determine a motion vectorand a reference picture index of the current prediction block by usingthe merging candidate list, the reference picture index of the currentprediction block specifying a reference picture of the currentprediction block, wherein the reference picture of the currentprediction block has a different temporal order from the current pictureand the collocated picture, obtain prediction samples of the currentprediction block by using reconstruction samples within the referencepicture specified by the reference picture index of the currentprediction block, the reconstruction samples being indicated by themotion vector of the current prediction block, obtain residual samplesof a current transform block by performing inverse-transform, andgenerate a reconstruct block by adding the prediction samples and theresidual samples, wherein while a reference picture index of the spatialmerging candidate is determined based on a reference picture of thespatial neighboring block, a reference picture index of the temporalmerging candidate is set to a fixed value of zero, the reference pictureindex of the temporal merging candidate being used to determine thereference picture index of the current prediction block, but not used todetermine the collocated picture of the current prediction block, andwherein the spatial neighboring block and the temporal neighboring blockare determined based on: a position of a coding block, a size of a blockon which a parallel merge processing is applicable, and a size of thecoding block, wherein the coding block includes the current predictionblock.
 2. The method of claim 1, wherein the temporal neighboring blockis representative one of a first block and a second block in thecollocated picture, the first block being located at a positioncorresponding to a bottom right position of the current prediction blockor the coding block, and the second block being located at a positioncorresponding to a central position of the current prediction block orthe coding block.
 3. The method of claim 1, wherein a temporal motionvector of the temporal merging candidate is derived by scaling a motionvector of the temporal neighboring block, based on both a first picturedistance and a second picture distance, wherein the first picturedistance is defined as a temporal order difference between thecollocated picture and a reference picture of the temporal neighboringblock, and wherein the second picture distance is defined as a temporalorder difference between the current picture and a reference picturespecified by the reference picture index of the temporal mergingcandidate.
 4. A method of decoding a video signal, comprising: selectinga collocated picture, the collocated picture having a different temporalorder from a current picture that includes a current prediction block,and the collocated picture comprising a temporal neighboring block whichis used to derive a temporal merging candidate of the current predictionblock; deriving a spatial merging candidate by using a spatialneighboring block; deriving the temporal merging candidate by using thetemporal neighboring block; generating a merging candidate listincluding the spatial merging candidate and the temporal mergingcandidate; determining a motion vector and a reference picture index ofthe current prediction block by using the merging candidate list, thereference picture index of the current prediction block specifying areference picture of the current prediction block, wherein the referencepicture of the current prediction block has a different temporal orderfrom the current picture and the collocated picture; and obtainingprediction samples of the current prediction block by usingreconstruction samples within the reference picture specified by thereference picture index of the current prediction block, thereconstruction samples being indicated by the motion vector of thecurrent prediction block; obtaining residual samples of a currenttransform block by performing inverse-transform; and generating areconstruct block by adding the prediction samples and the residualsamples, wherein while a reference picture index of the spatial mergingcandidate is determined based on a reference picture of the spatialneighboring block, a reference picture index of the temporal mergingcandidate is set to a fixed value of zero, the reference picture indexof the temporal merging candidate being used to determine the referencepicture index of the current prediction block, but not used to determinethe collocated picture of the current prediction block, and wherein thespatial neighboring block and the temporal neighboring block aredetermined based on: a position of a coding block, a size of a block onwhich a parallel merge processing is applicable, and a size of thecoding block, wherein the coding block includes the current predictionblock.
 5. The method of claim 4, wherein the temporal neighboring blockis representative one of a first block or a second block in thecollocated picture, the first block including a position correspondingto a bottom right position of the current prediction block or the codingblock, and the second block including a position corresponding to acentral position of the current prediction block or the coding block.